The present disclosure relates generally to videoconferencing systems, and more particularly, to managing the layout of multiple video streams displayed on a destination display screen during a videoconference.
Videoconferencing entails exchange of audio, video, and other information between at least two participants that are generally remotely located to one another. A videoconferencing endpoint is provided at each participant location to enable the participants to see and hear each other. The videoconferencing endpoint may include a camera for capturing video of local participants and a display device for displaying video of remote participants. The videoconferencing endpoint may also include a microphone for capturing audio signals from local participants and a speaker for converting audio signals received from the remote participants into sound. The videoconferencing endpoint may include additional display devices for displaying digital content. Where more than two endpoints participate in a videoconferencing session, a multipoint control unit (MCU) may be used to control the videoconference. The MCU may receive video, audio, and data from one or more endpoints, and then transmit the video, audio, and data to the appropriate endpoints, via a network.
FIG. 1 depicts a prior art videoconferencing system 100. The system 100 includes videoconferencing endpoints 101-105, a prior art multipoint control unit (MCU) 106, and a network 110. The network 110 may comprise a packet switched network, circuit switched network, or combinations thereof. The videoconferencing endpoints 101-105 may send and receive audio and video signals, and data. Network communication may be based on communication protocols, including H.320, H.324, H.323, and compression standards such as H.263, H.264.
The prior art MCU 106 initiates and manages videoconferencing sessions between two or more endpoints 101-105. The MCU 106 may mix audio data received from one or more endpoints 101-105, generate mixed audio data, and then send the mixed audio data to appropriate endpoints. Additionally, the MCU 106 receives one or more video streams from the endpoints and combines the video streams. The video streams are then sent by the MCU to the appropriate endpoints, to display the video streams on display screens of the endpoints.
The MCU 106 combines the video streams for display on the display screens based on a specified display layout. The display layout can be specified for various states and configurations of a videoconference call. For example, a near end display layout for a two-way call can include the video streams from a remote videoconferencing endpoint, while in a three-way videoconference call, the near end display may include various permutations and combinations of the video streams from both remote endpoints.
FIG. 2A is a flowchart for determining display layouts associated with destinations in various types of videoconferencing calls, performed by the prior art MCU 106. Changes in the state of the videoconferencing session are first detected, at step 120. For example, a change in state may be represented by a new videoconference call being initialed or if one or more endpoints joins or leaves an ongoing videoconference. If the MCU 106 detects a change in state, it then determines the type of ongoing videoconference, at step 122. If it is determined that ongoing the videoconference is a two-way videoconference, between a near end endpoint and a remote endpoint, then the MCU 106 generates two video streams, at step 124. One video stream is generated for a near end monitor MON and another video stream for a monitor at the remote endpoint EP1. The near end monitor MON receives the video stream from the remote endpoint EP1 (MON=EP1) and the remote endpoint EP1 receives the video stream from the near end endpoint MON (EP1=MON). Once it is decided where each video stream is presented, the presentation can be used to generate a display layout. The display layout specifies the spatial arrangement of the video stream, or streams, and any data stream, to be displayed on the display screens of each endpoint.
If it is determined that ongoing the videoconference is a three-way call, at step 126, the prior art MCU 106 determines the display layout and presentation, as shown in FIG. 2B. In FIG. 2B, video streams are routed between the near end monitor MON and various endpoints EP1, EP2 depending upon a current speaker (MON, EP1, EP2) 130 and a previous speaker 132, for each possibility of the current speaker. For example, if the current speaker is the near end speaker MON and the previous speaker was endpoint EP1, then the prior art MCU 106 routes the video stream from EP1 to the near end monitor MON and near end video of the current speaker MON is sent to the endpoints EP1 and EP2.
If it is determined that ongoing the videoconference is a 4-way call, at step 128 of FIG. 2A, then the number of video stream routing possibilities increases substantially, as illustrated in FIG. 2C. Video stream routing complexity is further increased with 5-way and 6-way, or higher, videoconference calls, hid tiding data content streams with the video streams during a videoconference also increases video stream routing complexity. Additionally, video stream routing complexity is increased when the multiple video streams and/or data streams are displayed at the same endpoint MON, EP1, EP2. As can be appreciated, video stream routing complexity becomes increasingly complex as the number of endpoints increases and as data streams are introduced into the videoconference.
If it is desired to change any portion of the spatial arrangement of the specified display layout of the video stream, or streams, and/or data stream displayed on the display screens of the endpoints, the changes are performed at the code level, to allow the prior art MCU 106 to generate the desired display layouts. Changes to the code necessitate recompiling of the code for the changes to take effect. Incorporating changes and recompiling the code is often beyond the skills of most end users. Additionally, modifications to the code may inject bugs or other errors into the system, which may be hard to detect and cause multiplicative ill-effects to the system.